Skip to content

sync: hand the RWMutex over instead of a re-test of the reader count - #5630

Open
yohimik wants to merge 1 commit into
tinygo-org:devfrom
yohimik:upstream-pr/sync-rwmutex
Open

sync: hand the RWMutex over instead of a re-test of the reader count#5630
yohimik wants to merge 1 commit into
tinygo-org:devfrom
yohimik:upstream-pr/sync-rwmutex

Conversation

@yohimik

@yohimik yohimik commented Aug 30, 2026

Copy link
Copy Markdown

sync: hand the RWMutex over instead of a re-test of the reader count

What this does

sync.RWMutex counts the readers that hold the lock and the readers that queue
behind a waiting writer in one number, and both sides wait on a predicate over
that number. Two interleavings stop the program permanently.

  1. A writer waits until the count shows no readers at all. A reader that arrives
    during that wait joins the same count, so the last holder of the lock no
    longer sees the condition that wakes the writer.
  2. A reader that Unlock releases reads the count again instead of an acquire.
    A writer that arrives in between changes the base of the count, so the reader
    goes back to sleep after its wakeup is spent, while that writer waits for it.

Both cases end with every party parked and nobody left to wake anyone.

The change adopts the split that the Go standard library uses. A writer records
how many readers it finds and waits only for those, so later readers cannot
starve it. Counting semaphores hand the lock over, so a released waiter holds
the lock and does not test a value again that a third party can change back.
internal/task.Semaphore cannot do this, because one Post does nothing when
there are several waiters, so src/sync/mutex.go gets a small futex semaphore
that can.

Why it matters

Ordinary code reaches this. syscall.ForkLock is an RWMutex, os.Pipe
read-locks it and os.StartProcess write-locks it, so a program that starts
processes and makes pipes at the same time can stop.

Evidence

Two regression tests are added to src/sync/mutex_test.go. sync is already in
TEST_PACKAGES_FAST, so they run on every platform that the standard library
test matrix covers, with no makefile change.

Measured on macOS 26.6 arm64 with tinygo test sync.

Test Current dev With this change
TestRWMutexWriterNotStarvedByLateReaders FAIL after 10.11s, "the writer did not wake after the last reader unlocked" PASS in 0.10s
TestRWMutexHandoffToQueuedReaders FAIL after 30.00s, "readers and writers stopped while they hand the lock over" PASS in 0.00s

The same shape was seen with real work. A probe that starts sixteen processes
at the same time and gives each one a pipe stopped within twelve seconds on
every run before this change, and completed on twenty runs out of twenty after
it, on linux and on macOS.

A downstream product also ships binaries built with a fork that carries this
change, in the published release dispat v1.4.0.
https://github.com/yohimik/dispat/releases/tag/services%2Fdispat%2Fv1.4.0

Scope

  • No API change.
  • The cooperative, threads and cores schedulers all use internal/task.Futex,
    so no path is special-cased.
  • Windows, wasm and baremetal targets get the same code and the same tests.

Notes for review

  • rwSem is small on purpose. If the maintainers would rather have a counting
    semaphore in internal/task next to Semaphore, it can move there.
  • Unlock now reports "sync: Unlock of unlocked RWMutex" through
    runtimeFatal, which the old code did not do. Say if that check is unwanted.

Related pull requests

Each open PR in this series has a separate change. A dependency is not a copied commit.

In tinygo-org/net

Full Darwin networking also needs the merged net changes and a later src/net pin update. No upstream merge or current full-suite pass is implied by this list.

Additional independent gaps found in Crier are covered by #5655 (cookie-jar loader merge) and tinygo-org/net#82 (ListenConfig). They do not duplicate the process, TLS, deadline or server-TLS changes above. Current Crier comparison work is separate from its standard-Go release.

Current Crier evidence

The Crier report separates the original size comparison from a new CI-built candidate test. The original stripped Linux ARM64 result is 13,829,248 bytes versus Go's 30,277,794 bytes. Two render fixtures exceed pixel tolerance.

Candidate e7d34c8c126e0eecd2ce711915f2f88d112d833a, with net 0f460803, passed all 24 fork CI checks. Its downloaded compiler artifacts, with no source overlays, build unchanged Crier 7edaff9. Linux ARM64 E2E passed 143 cases with no failures or skips. Darwin ARM64 passed 142 with no failures and one platform trust-store skip. Darwin startup now works. A separate Darwin local-TLS matrix passes certificate rejection, plaintext refusal, update to a Go 1.1.0 target, and offline rollback. Both platforms pass spawn, signal, cookiejar, os/fcntl, and network probes.

These are combined-candidate results, not proof that this PR alone supplies all features. No new pixel or stripped-size comparison, amd64 execution, or current Dispat acceptance is claimed. WaitDelay and the recorded net limits remain open. No fork release was published.

shibukawa added a commit to shibukawa/tinygodriver that referenced this pull request Sep 2, 2026
…on the TinyGo path

TinyGo's sync.RWMutex (through at least 0.42) deadlocks whenever a reader
arrives while a writer is waiting for the existing readers to drain. Lock()
subtracts rwMutexMaxReaders and waits for the last RUnlock to bring the count
back to exactly that; RLock() adds 1 and then waits for the count to turn
positive, keeping its +1 while it waits. A reader that arrives mid-wait is
therefore counted as a holder that never leaves: the writer is never woken and
the reader waits for the writer's Unlock. Standard Go snapshots the readers a
writer must wait for in a separate counter. Upstream fix: tinygo-org/tinygo#5630,
open at time of writing.

This is what `tinygo test ./websocket` had been hanging on, roughly one run in
three: a stack sample showed 15 threads in RWMutex.RLock and one in
RWMutex.Lock, all reached through netdev.Device.mu, which every Send and Recv
read-locks while Socket, Accept and Close write-lock it. A four-step
interleaving reproduces it deterministically; 16 readers and a writer around a
map hung 13 of 13 runs; the same netdev echo probe went from 11 of 40 hung to
0 of 40 with a plain mutex.

internal/syncx.RWMutex is sync.RWMutex on standard Go and a plain sync.Mutex
behind the same method set on tinygo and force_tinygo_logic. Every RWMutex a
tinygo build could reach now uses it: netdev's socket table and the darwin and
windows TLS session tables, httpmux, the dynamodb and datastore field caches,
the s3 region, fasthttp's HostClient map (as vendor.py patches, PATCHES.md
section 8) and the five registries in the mysql fork (PETITWEB comments,
README). Each critical section is a map lookup, so serializing readers costs
nothing measurable.

Three tests pin it down. TestRWMutexHandsOverToWaitingWriter runs the four-step
interleaving against the shim on every build. TestUpstreamRWMutexStillDeadlocks
(tinygo only) runs it against sync.RWMutex and expects the deadlock, so it
fails the day a TinyGo release ships the fix, which is the signal to retire the
shim. TestNoStdRWMutexOnTinyGoPath asks `go list -tags tinygo` for the files
of every package on darwin, linux and windows and parses them for the
selector, so neither first-party code nor a re-vendored fork can drift back.

Verified: go test and go test -tags force_tinygo_logic across the module;
tinygo test for syncx, netdev, httpmux, httprevproxy, fasthttp and
fasthttpwebsocket; `tinygo test ./websocket` 20 times under a watchdog with
0 hangs, against 5 of 16 before; the examples that touch the swapped packages
build, as does a linux/arm64 cross-link; mingw vets the windows session table.
TestLargeMessage's occasional EPIPE, seen on the untouched tree as well, is a
separate flake and is not changed by this.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
shibukawa added a commit to shibukawa/tinygodriver that referenced this pull request Sep 2, 2026
TinyGo's sync.RWMutex deadlocks whenever a reader arrives while a writer is
waiting; netdev.Device.mu is read on every Send and Recv and written on every
Socket, Accept and Close, which is how tinygo test ./websocket hung one run in
three. internal/syncx.RWMutex is the standard type on standard Go and a plain
mutex on TinyGo, every reachable RWMutex now uses it, a policy test keeps it
that way, and a tinygo-only test fails the day a release ships the upstream
fix (tinygo-org/tinygo#5630). 0 hangs in 20 runs, against 5 in 16.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
RWMutex counts the readers that hold the lock and the readers that queue
behind a waiting writer in one number, and both sides wait on a predicate over
that number. Two interleavings stop the program permanently.

A writer waits until the count shows no readers at all. A reader that arrives
during that wait joins the same count, so the last holder of the lock no
longer sees the condition that wakes the writer.

A reader that Unlock releases reads the count again instead of an acquire. A
writer that arrives in between changes the base of the count, so the reader
goes back to sleep after its wakeup is spent, while that writer waits for it.

Use the split that the standard library uses. A writer records how many
readers it finds and waits only for those, so later readers cannot starve it.
Counting semaphores hand the lock over, so a released waiter holds the lock
and does not test a value again that a third party can change back.
task.Semaphore cannot do this, because one Post does nothing when there are
several waiters, so the file gets a small futex semaphore that can.

Ordinary code reaches this. syscall.ForkLock is an RWMutex, os.Pipe read-locks
it and os.StartProcess write-locks it, so a program that starts processes and
makes pipes at the same time can stop.

The two new tests fail on the current code and pass with this change.
@yohimik
yohimik force-pushed the upstream-pr/sync-rwmutex branch from 7001593 to 6965ab2 Compare September 2, 2026 08:49
@yohimik

yohimik commented Sep 2, 2026

Copy link
Copy Markdown
Author

Rebased on dev after the 0.42.0 release. The problem is present in v0.42.0 as
released. The two new tests were put into the src/sync tree of the official
v0.42.0 tarballs and run with that toolchain. --- FAIL: TestRWMutexWriterNotStarvedByLateReaders (10.10s), "the writer did not wake
after the last reader unlocked", on darwin/arm64 and linux/arm64 in every run.
TestRWMutexHandoffToQueuedReaders is a race. It failed in 3 of 8 darwin runs.

yohimik added a commit to yohimik/tinygo that referenced this pull request Sep 5, 2026
Use posix_spawn on hosted Linux and Darwin. Map process files, apply the
working directory and process group, and clear the child signal mask.
Use wait4 for process status and support Kill and Signal.

Mark Darwin pipes close-on-exec under ForkLock. Darwin also needs the
fcntl wrapper in PR tinygo-org#5612 and the libSystem symbols in PR tinygo-org#5636.
Concurrent spawn and pipe creation need the RWMutex fix in PR tinygo-org#5630.

Keep the process stubs on other targets and add process regression tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant